Ability Level Estimation of Students on Probability Unit via Computerized Adaptive Testing

Problem Statement: Learning-teaching activities bring along the need to determine whether they achieve their goals. Thus, multiple choice tests addressing the same set of questions to all are frequently used. However, this traditional assessment and evaluation form contrasts with modern education, where individual learning characteristics are featured. Hence, the use of Computerized Adaptive Testing (CAT) systems, which set the difficulty level in accordance with the ability levels of individuals, is spreading. However, these systems are not prevalent in Turkey. Therefore, it is important to develop and assess a CAT system to be integrated into Turkish curriculum. Purpose of Study: The purpose of this study is to develop a CAT system that can be used in the sub-levels of Turkish curricula (high school) and assess it in terms of reliability. Methods: In this study, a CAT system aimed at the sub-topics (permutation, combination, binomial expansion, and probability) of the unit of probability covered in 11th grade mathematics was developed. Estimation sensitivity of the developed CAT system, from lower to higher ability levels, was assessed. To this end, an exam was implemented in a high school located in Trabzon in the 2011-2012 fall semester. Eighty-four 11th graders participated in the implementation. Using the data derived from the implementation, reliability coefficient values of each learner were calculated. System records were used for data collection. Results: Reliability coefficient values for each student in 5 different exams permutation, combination, binomial expansion, probability, and end-ofunit were calculated. Findings of the study indicated that the developed * Dr. Department of Software Engineering, Karadeniz Technical University, Trabzon. e-mail: [email protected] ** Dr. Department of Software Engineering, Karadeniz Technical University, Trabzon. e-mail: [email protected] 28 Hacer Özyurt, & Özcan Özyurt CAT system produced tests with high reliability for all subjects. Average reliability coefficient values for each subject were found as 0.93, 0.93, 0.88, 0.93, and 0.91. Discussion and Conclusion: By addressing questions from lower to higher levels, which are appropriate to the ability level of each learner, CATs increase sensitivity and reliability of measurement. The developed CAT system addresses questions appropriate for the ability level of the learner, which increases sensitivity in measurement. In addition, it was observed that CAT systems developed for exams have the characteristic of making sensitive measurements ranging from lower to higher levels. This study, in relation to UZWEBMAT-CAT assessment, proved that CAT systems can safely be used within Turkish curriculum.